Task placement of parallel multi-dimensional FFTs on a mesh communication network

For many scientific applications, the Fast Fourier Transformation (FFT) of multi-dimensional data is the kernel which limits scalability to large numbers of processors. This paper investigates an extension of a traditional parallel threedimensional FFT (3D-FFT) implementation. The extension within a parallel 3D-FFT consists of customized MPI task mappings between the virtual processor grid of the algorithm and the physical hardware of a system with a mesh interconnect. Consequentially, we derived a simple model for the scope of performance of a large class of mappings on the basis of bandwidth considerations. This model enables us to identify scaling bottlenecks and hotspots of parallel, communication intensive 3D-FFT applications when MPI tasks are mapped in the default way onto the network. The predictions of the model are tested on an IBM eServer Blue Gene/L system. The results demonstrate that a carefully chosen mapping pattern with regards to the network characteristics yields significant improvement.